The present invention relates to video cameras, and in particular to video cameras connected to a computer or other intelligent device by a shared interface, such as the universal serial bus (USB).
Video cameras for providing both still pictures and motion pictures have been developed which can connect to a computer to provide a digitized image. Such cameras can be connected to the computer via a shared bus, such as the USB. This limits the bandwidth of data that can be sent, requiring compression of the video data. Typically, cameras will do some of the preprocessing of the image in the camera electronics, thus both off-loading processing from the CPU of the computer and potentially reducing the amount of data that has to be compressed and sent over the bus.
With such a design, some functions are required to be done in the camera. For example, a CCD or CMOS sensor in the camera may have defective detector positions which do not properly collect light. These can be compensated for by averaging the amount of light detected at neighboring detector sites and substituting this for the value provided by the defective site. This type of processing needs to be done before other types of processing which are now typically done in a video camera, such as color processing and scaling.
FIG. 1 is a block diagram of a typical video camera which can connect over a bus to a host. A lens 10 focuses light onto a sensor 12 with associated control logic for reading out the sensor data. The data is provided to an analog-to-digital converter (ADC) 14, where it is digitized. The data is then stored in a frame buffer memory 16. Pixel correction can be applied to the data with a circuit 18. Correction data may be stored in a programmable memory 20. This memory may be programmed during the manufacturing process, before the camera is even shipped.
A number of other functions are typically performed in the camera, shown in FIG. 1 in a particular order, although they may be in other orders or connected via a bus to vary when they are invoked. For example, a color processing circuit 22 may perform a number of color functions, such as converting the received data from one color format, such as YUV, to another format, such as RGB. A scaler function 24 can scale the image to fit the display desired at the host. A compression circuit 26 will compress the data to reduce the amount of data to be transferred over the shared bus. Finally, a bus interface circuit 28 implements the protocols of a bus 30 for transferring data to a remote host.
Huffman coding is one of the algorithms most widely used to compress video data. Huffman coding has been used in various image and video compression standards such as the standards specified by the Joint Photographic Experts Group (JPEG), Motion Picture Experts Group (MPEG), MPEG-II, H.261, H.263, and H.323. These standards are included herein by reference, in there entirety and for all purposes. Huffman coding is a lossless entropy coding technique. Huffman coding utilizes probability to select the shortest code word for the data that occur most frequently. For example, when encoding English text, “E” which is a very popular character in the English language can be represented by a 2-bit code word, whereas “A” which is not used as frequently can be represented by an 8-bit code word.
As a result, Huffman coding compresses data by assigning shorter code words to more frequently occurring data and longer code words to less frequently occurring data. The assigned code words can be maintained in a table which is then used by both recipients and senders. Further details regarding Huffman encoding can be found in “A Method for the Construction of Minimum Redundancy Codes,” Proceedings of the IRE, Vol. 40, No. 9, September 1952, pp. 1098–1101, by D. A. Huffman, which is incorporated herein by reference in its entirety and for all purposes.
Moreover, Huffman code words are constructed in such a way that no code word is the prefix of another code word. This assures the unambiguous decoding of each symbol from a bitstream of data despite the variable lengths of the codes. Additional description of the characteristics of Huffman codes can be found in Data Compression Book, M&T Books, 1996, by M. Nelson and J. L. Gailly, which is also incorporated herein by reference in its entirety and for all purposes. On pages 31–35, this reference proposes a decoding algorithm based on tree-tracing. This algorithm is, however, not suitable for fast decompression using most of the currently available microprocessors such as Intel®'s Pentium® family.
The vignetting effect is when the brightness around the edges of a screen is less than the brightness in the center of the screen, creating a halo effect. Professional photographers will sometimes try to intentionally create this effect with an appropriate lens or screen, to produce a softened image at the edges of a picture. However, when this effect appears unintentionally in digital photographs, it is annoying. A variety of techniques have been developed to measure and address vignetting effects.
U.S. Pat. No. 4,884,140 shows an analog circuit for providing vignetting compensation for a video camera using a zoom lens which causes vignetting.
U.S. Pat. No. 5,434,902 shows measuring the vignetting effect for an x-ray examination apparatus by using a constant brightness image. A correction factor is then stored in the memory for each pixel.
U.S. Pat. No. 5,576,797 shows the detection of vignetting effects in a camera with a focus detecting device.
U.S. Pat. No. 5,381,174 shows the correcting of vignetting due to operation of a zoom lens by using a field frequency sawtooth signal.
U.S. Pat. No. 4,816,663 shows the detection of vignetting between a photo taking lens and focus detecting optical system.
Digital cameras for use with a personal computer for personal teleconferencing have become cheaper and cheaper. This puts pressure on the camera manufacturers to use cheaper lenses, which have more of a vignetting effect. In addition, the price pressure forces manufacturers to use fewer semiconductor chips, thus making the use of semiconductor memory to store vignetting corrections undesirable. Accordingly, there is a need for a digital camera which is inexpensive to manufacture and also corrects for vignetting of an inexpensive lens.
The processes used to produce image sensors, though highly advanced, are not perfect. Photosites in an image sensor array can vary in their sensitivity or ability to convert light into electric currents. Some variation is expected and tolerable, but on occasion, individual photosites in an array can be dead, inactive or over active. This can result in a displayed image having a dark or black dot, bright white dot or incorrect color dot, all of which are unwanted and are undesirable to the end user. Those in the digital imaging field have also recognized the problems caused by improper digitization of target images and have proposed various solutions to detect and correct defective pixels. These prior art methods and systems can be broadly classified into two groups.
On one hand, there are various hardware-based methods for detecting and correcting defective pixels taken from an image sensor array. These hardware-based methods and systems are quite common to video camera manufacturers. Most conventional video cameras using a solid state image pickup device, incorporate a defective pixel detection and correction circuit for correcting defective pixels taken from an image sensor array. The defective pixels are produced when or after the image sensor array is manufactured. In such a camera, a defective pixel correction circuit detects a defective pixel and stores the position data and various data related to the defective pixel in a read only memory (ROM) or the like. Then, when the video camera is in use, pixel data from the defective pixel is replaced by data from a pixel near the defective pixel. One such method and system is disclosed in U.S. Pat. No. 5,796,430. The disadvantage of such approaches is the need to incorporate and program memory devices during product assembly and testing which add costs and delays. Additionally, the hardware device costs will also increase due to the need to add the correcting circuitry or logic to the video processing application specific integrated circuit (ASIC).
On the other hand, there are various software-based methods for detecting and correcting defective pixels in digital images. One such method and system is disclosed in U.S. Pat. No. 5,982,946. Such software-based methods are generally aimed at correcting bad image pixels in an already digitized and fully processed image. These already digitized images are in their final displayable and stored forms which have already been through color processing, compression and various other processing that are performed on the raw data which is read off an image sensor array. Therefore, such software-based methods which treat completely processed digital images, may correct anomalous pixels that are caused by any number of artifacts such as dust or dirt on the original scene, dust or dirt on the lens which was used to capture the scene as well as anomalous pixels in the digitized image which were caused by defective photosites. These methods typically rely on rather sophisticated and expensive systems and computer software to detect and correct bad pixels. These methods generally require user input to detect the location of the potentially bad image pixels. Once the user has visually scanned an entire image and flagged potentially bad image pixels, the system incorporating the software program takes over to correct the flagged defective pixels. Besides requiring user input, such approaches are also expensive, tedious and very time consuming.
The proliferation of inexpensive PC-interfacing digital still and video camera devices requires rapid, dynamic, inexpensive and intelligent defective pixel detection and correction solutions.